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Objective:  We  describe  a  novel  concept,  situation 
awareness  recovery  (SAR),  and  we  identify  perceptual 
and  cognitive  processes  that  characterize  SAR. 

Background:  Situation  awareness  (SA)  is  typically 
described  in  terms  of  perceiving  relevant  elements  of  the 
environment,  comprehending  how  those  elements  are 
integrated  into  a  meaningful  whole,  and  projecting  that 
meaning  into  the  future.  Yet  SA  fluctuates  during  the  time 
course  of  a  task,  making  it  important  to  understand  the 
process  by  which  SA  is  recovered  after  it  is  degraded. 

Method:  We  investigated  SAR  using  different  types 
of  interruptions  to  degrade  SA.  In  Experiment  I,  par¬ 
ticipants  watched  short  videos  of  an  operator  perform¬ 
ing  a  supervisory  control  task,  and  then  the  participants 
were  either  interrupted  or  not  interrupted,  after  which 
SA  was  assessed  using  a  questionnaire.  In  Experiment 
2,  participants  performed  a  supervisory  control  task  in 
which  they  guided  vehicles  to  their  respective  targets 
and  either  experienced  an  interruption,  during  which 
they  performed  a  visual  search  task  in  a  different  panel, 
or  were  not  interrupted. 

Results:  The  SAR  processes  we  identified  included 
shorter  fixation  durations,  increased  number  of  objects 
scanned,  longer  resumption  lags,  and  a  greater  likelihood 
of  refixating  on  objects  that  were  previously  looked  at. 

Conclusions:  We  interpret  these  findings  in  terms 
of  the  memory-for-goals  model,  which  suggests  that 
SAR  consists  of  increased  scanning  in  order  to  com¬ 
pensate  for  decay,  and  previously  viewed  cues  act  as 
associative  primes  that  reactivate  memory  traces  of 
goals  and  plans. 

Keywords:  attention,  supervisory  control,  eye  move¬ 
ments,  memory  for  goals,  situation  awareness 
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INTRODUCTION 

Pilots,  automobile  drivers,  power  plant 
workers,  and  supervisory  control  operators  are 
required  to  allocate  attention  to  multiple  objects 
that  must  be  monitored  concurrently.  Yet  human 
attention  is  limited,  resulting  in  the  need  to  con¬ 
stantly  shift  attention  between  different  objects 
in  these  dynamic  tasks.  These  shifts  in  atten¬ 
tion  result  in  moment-to-moment  fluctuations 
in  situation  awareness  (SA).  Take  an  example 
from  a  multirobot  supervisory  control  task:  If 
one  vehicle  needs  help  because  it  is  on  a  col¬ 
lision  course,  the  operator  needs  to  attend  to 
the  vehicle  in  a  timely  manner.  He  or  she  must 
perceive  the  relevant  objects,  analyze  the  state 
of  the  objects,  make  a  plan,  and  execute  the 
plan  by  directing  the  vehicle  to  change  course. 
During  this  time,  the  operator’s  attention  is 
primarily  on  a  single  vehicle,  and  the  operator 
may  be  unaware  of  or  may  forget  other  vehicles 
that  need  help.  After  such  reductions  in  SA,  a 
skilled  operator  must  reassess  the  environment 
to  recover  SA;  we  call  this  process  situation 
awareness  recovery  (SAR). 

Background 

Endsley  (1995a)  defined  SA  as  consisting  of 
three  stages:  the  perception  of  elements  in  an 
environment,  the  integration  of  those  elements 
into  a  comprehensible  meaning,  and  the  projec¬ 
tion  of  that  meaning  into  the  future.  When  two 
aircraft  are  on  a  collision  course,  the  operator 
must  notice  the  location  and  state  of  the  vehicles 
(perception  stage  of  SA),  aggregate  that  percep¬ 
tion  information  to  evaluate  the  situation  (com¬ 
prehension  stage  of  SA),  and  project  that  situa¬ 
tion  onto  future  events  (projection  stage  of  SA). 
Loss  of  SA  is  often  strongly  correlated  with 
degraded  task  perfonnance  (Durso  et  al.,  1995; 
Endsley,  1995b),  constituting  the  main  cause 
of  aircraft  accidents  in  an  analysis  of  over  200 
incidents  (Hartel,  Smith,  &  Prince,  1991;  Jones 
&  Endsley,  1996);  however,  SA  is  distinct  from 
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performance  in  that  it  precedes  decision  making 
and  is  not  strongly  correlated  with  performance 
in  simple  tasks  (Endsley,  1995a). 

Endsley’s  three-level  model  describes  SAas  a 
product,  whereas  other  models  emphasize  the 
processes  that  underlie  SA.  For  example,  the  per¬ 
ceptual  cycle  model  (Smith  &  Hancock,  1995) 
highlights  the  cognitive  processes  involved  in  SA 
and  the  interaction  between  these  processes  and 
the  environment  (for  a  review,  see  Salmon,  Stan¬ 
ton,  Walker,  &  Green,  2006).  Also,  in  Endsley’s 
( 1 995b)  seminal  paper,  several  processing  mech¬ 
anisms  were  hypothesized  to  impact  SA,  includ¬ 
ing  working  memory,  attention  distribution, 
goal-directed  processing,  mental  models,  sche¬ 
mata,  and  automaticity.  Indeed,  Sohn  and  Doane 
(2004)  found  that  working  memory  capacity  was 
predictive  of  SA  performance,  especially  for 
novices,  and  Gugerty  and  Tirre  (2000)  found  a 
correlation  between  SA  and  working  memory, 
perceptual  motor  ability,  visual  processing  abil¬ 
ity,  and  temporal  processing  ability.  Although 
these  studies  show  the  cognitive  abilities  that  are 
involved  in  SA,  they  do  not  demonstrate  the 
mechanism  by  which  these  processes  operate. 

In  order  to  extend  Endsley’s  model  by  obtain¬ 
ing  a  fine-grained  process  understanding  of  how 
SA  unfolds  and  fluctuates  over  the  time  course 
of  a  task,  we  introduce  and  investigate  a  new 
concept,  SAR.  SAR  is  defined  as  the  process  of 
restoring  SA  after  SA  has  been  reduced.  A  reduc¬ 
tion  in  SA  can  be  caused  by  a  loss  of  attention, 
interruptions,  multitasking,  operator  overload, 
and  so  on.  Since  interruptions  can  adversely 
affect  SA,  it  is  hypothesized  that  the  operator 
will  be  more  likely  to  engage  in  SAR  after  an 
interruption.  If  distinct  perceptual  and  cognitive 
processes  characterize  SAR,  then  these  pro¬ 
cesses  can  be  identified  and  incorporated  within 
a  process  account  of  SA. 

There  are  subtle  distinctions  among  the  ways 
in  which  interruptions  impact  SA.  As  described 
by  Boehm-Davis  and  Remington  (2009),  inter¬ 
ruptions  can  differ  based  on  the  trigger  of  the 
interruption  (endogenous  versus  exogenous); 
the  disengagement  from  the  primary  task — 
whether  the  secondary  task  can  be  delayed  or 
else  needs  immediate  attention;  and  how  the  pri¬ 
mary  task  is  resumed — whether  the  primary  task 


is  dynamically  changing  during  the  interruption  or 
else  is  static.  McFarlane  (2002)  distinguished 
between  interruptions  that  are  immediate,  whereby 
the  operator  does  not  control  when  he  or  she  is 
interrupted,  and  those  that  are  negotiated,  whereby 
the  operator  can  control  when  he  or  she  is 
interrupted. 

Trafton,  Altmann,  Brock,  and  Mintz  (2003) 
presented  a  task-analytic  framework  for  interrup¬ 
tions  in  which  the  ability  to  resume  after  an  inter¬ 
ruption  is  affected  by  the  interruption  lag  and  the 
type  of  interruption.  The  interruption  lag  is  the 
time  spent  preparing  for  an  interruption  before  it 
occurs.  The  type  of  interruption  affects  the  ability 
to  resume  after  an  interruption  because  a  second¬ 
ary  task  that  requires  more  cognitive  demands 
limits  the  rehearsal  of  the  primary  task.  Based  on 
this  framework,  exogenous  and  immediate  inter¬ 
ruptions  can  be  particularly  damaging  to  SA 
because  prospective  goal  encoding  (i.e.,  “What 
was  I  about  to  do?”;  Brondimonte,  Einstein,  & 
McDaniel,  1996)  and  retrospective  memory 
rehearsal  (i.e.,  “What  was  I  doing?”)  are  inhib¬ 
ited  in  these  circumstances.  Indeed,  a  longer 
interruption  lag  resulted  in  a  shorter  resumption 
lag  (Trafton  et  al.,  2003).  Additionally,  reduction 
in  the  cognitive  demands  of  the  secondary  task 
mitigated  the  negative  effects  of  an  interruption 
(Monk,  Boehm-Davis,  &  Trafton,  2004). 

We  argue  here  that  after  an  interruption,  SA  is 
lost  due  to  a  specific  set  of  processes.  Memory 
traces  decay  because  the  operator  is  involved  in 
another  task,  which  results  in  the  inability  to 
detect/remember  changes  in  the  environment 
(Altmann  &  Gray,  2008;  Hodgetts  &  Jones, 
2006;  Monk  et  al.,  2004;  Ratwani,  McCurry,  & 
Trafton,  2008;  Trafton  et  al.,  2003).  If  the  mem¬ 
ory  trace  was  not  well  encoded,  the  interruption 
will  have  a  more  detrimental  impact  on  SA. 
Although  much  is  known  about  the  impact  of  SA 
on  task  performance,  the  components  of  SA,  and 
the  cognitive  processes  involved  in  SA,  it 
remains  unclear  how  participants  recover  SA 
after  it  is  degraded. 

Memory  for  Goals  (MFG)  Theory 
of  SAR 

The  MFG  model  (Altmann  &  Trafton,  2002) 
offers  a  promising  framework  for  characterizing 
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the  process  of  SAR.  The  MFG  model  is  based  on 
the  theoretical  construct  of  activation  of  memory 
elements — in  particular,  activation  as  construed  in 
the  ACT-R  (Adaptive  Control  of  Thought-Ratio¬ 
nal)  cognitive  architecture  (Anderson  et  al.,  2004). 
The  MFG  model  was  initially  used  to  interpret 
how  subgoals  are  suspended  and  resumed  in  a 
problem-solving  task  (Altmann  &  Trafton,  2002). 
MFG  can  also  be  applied  to  a  task  like  supervisory 
control,  wherein  each  vehicle  represents  a  sepa¬ 
rate  subgoal  that  must  be  perceived  and  retained 
in  order  for  the  situation  to  be  comprehended  and 
analyzed.  MFG  has  since  been  used  to  explain  the 
disruptive  effects  of  a  task  interruption  (Altmann 
&  Trafton,  2007),  and  it  has  been  used  to  explain 
sequence  errors  (Ratwani  &  Trafton,  2011;  Traf¬ 
ton,  Altmann,  &  Ratwani,  2011). 

Two  constraints  of  the  MFG  model  make  spe¬ 
cific  predictions  regarding  SAR:  (a)  the  strength¬ 
ening  constraint  and  (b)  the  priming  constraint 
(Altmann  &  Trafton,  2002).  The  strengthening 
constraint  includes  mechanisms  by  which  mem¬ 
ories  for  goals  are  established  and  reinforced. 
Specifically,  the  frequency  and  recency  of  goal 
retrievals  strengthen  the  activation  of  the  goal  in 
memory.  For  example,  working  on  a  vehicle  in  a 
supervisory  control  simulation  will  strengthen 
the  activation  of  the  goal  associated  with  that 
vehicle.  The  current  most  active  goal  is  the  one 
most  likely  to  be  retrieved  and  thus  to  direct 
behavior.  Over  time,  the  activation  of  a  goal 
decays  unless  it  is  reactivated.  Therefore,  a  goal 
retrieval  request  is  more  likely  to  return  a 
recently  and  more  frequently  sampled  goal.  The 
need  to  reactivate  goals  suggests  that  during 
SAR,  the  operator  will  preferentially  sample 
previously  sampled  objects  rather  than  previ¬ 
ously  unsampled  objects,  thereby  reactivating 
recent  goal  traces  that  have  decayed. 

The  second  constraint,  the  priming  constraint 
of  the  MFG  model,  can  be  leveraged  to  make 
predictions  about  SAR.  The  priming  constraint 
posits  that  contextual  retrieval  cues,  such  as  a 
vehicle’s  visual  representation,  can  provide 
associative  activation  or  priming  that  can  boost 
the  activation  of  a  goal.  In  the  MFG  model,  as  an 
operator  works  on  a  vehicle,  that  event  is 
encoded  as  an  episodic  code  that  can  decay  dur¬ 
ing  an  interruption.  This  decay  causes  reduced 
SA  and  a  need  for  the  operator  to  make  the  addi¬ 


tional  step  of  examining  the  environment  to 
refresh  the  activation  of  decaying  episodic 
codes.  A  result  of  this  extra  recovery  step  is  that 
it  will  take  more  time  to  perform  a  task  having  a 
task  suspension  than  a  task  with  no  suspension 
(Trafton  et  al.,  2003).  This  increase  in  time  after 
an  interruption  is  known  as  the  resumption  lag. 
In  other  words,  after  an  interruption,  the  opera¬ 
tor  is  less  familiar  with  the  environment  (i.e.,  has 
reduced  SA)  because  the  episodic  codes  for  the 
situation  and  environment  have  decayed  and  so 
attends  to  cues,  which  results  in  taking  a  longer 
amount  of  time  to  recover  his  or  her  awareness 
than  without  interruption. 

Combining  activation,  decay,  and  priming, 
MFG  thus  yields  the  following  predictions  of 
where  the  operator’s  attention  will  be  drawn  in 
order  to  improve  SA.  When  the  activation  of 
memory  elements  is  low,  such  as  when  decay 
has  occurred  after  an  interruption,  the  operator 
must  engage  in  SAR  in  order  to  reinstate  the  pre¬ 
vious  context  and  recover  SA.  In  a  simple  model, 
one  might  imagine  a  system  whereby  the  opera¬ 
tor  nondiscriminately  samples  across  goals 
based  on  bottom-up  cues,  such  as  saliency.  Yet 
in  the  MFG  theory,  priming  acts  as  the  mecha¬ 
nism  by  which  previous  goals  are  reinstated  and 
SA  is  restored.  In  other  words,  after  a  period  of 
low  SA,  when  there  is  partial,  though  decayed, 
memory  of  the  situation,  MFG  predicts  that  top- 
down  location-based  memory  cues  prime  the 
goal  state.  These  cues  are  dependent  on  the  goal 
state.  Thus  if  the  objective  of  the  task  is  to  pre¬ 
vent  vehicles  from  intersecting  with  hazards,  the 
effective  cues  are  the  objects  (vehicle,  hazard, 
target)  involved  in  the  intersect  scenario.  MFG 
thus  predicts  that  following  an  interruption, 
operators  will  engage  in  SAR  by  attending  to 
previously  attended  objects,  which  prime  and 
therefore  raise  the  activation  of  previous  goals 
and  plans.  In  effect,  these  old  objects  are  acting 
as  contextual  cues  that  promote  the  recovery  of 
situation  awareness. 

Although  this  scenario  is  consistent  with  Ens- 
ley’s  model  for  goal-directed  processing  (Ends- 
ley,  1995b),  Endsley’s  model  describes  how 
goals  impact  information  processing  at  a  higher 
level  than  MFG.  Because  MFG  describes  how 
goals  decay  and  are  activated  over  a  fine-grained 
time  course,  it  can  make  explicit  predictions 
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about  how  contextual  cues  can  be  strategically 
used  by  the  operator  to  improve  SA. 

The  integrated  framework  for  maintaining 
and  recovering  SA  proposed  by  St.  John  and 
Smallman  (2008)  involves  concepts  similar 
to  the  MFG  model  to  make  predictions  about 
where  attention  will  be  allocated.  St.  John  and 
Smallman  organized  SA  into  four  stages:  (a) 
change  detection  before  an  interruption,  (b)  pre¬ 
paring  for  an  interruption,  (c)  reorientation  after 
an  interruption,  and  (d)  change  detection  after  an 
interruption.  Similar  to  the  SAR  prediction  from 
the  MFG  model  that  participants  look  at  contex¬ 
tual  cues  when  recovering  SA,  the  integrated 
framework  includes  an  additional  reorientation 
step  following  an  interruption.  Yet  when  there  is 
no  interruption,  the  goal  state  is  less  degraded, 
and  there  is  less  need  to  add  the  additional  step 
of  attending  to  a  context  cue  (i.e.,  reorienting). 
Whereas  the  integrated  framework  is  relatively 
descriptive,  the  MFG  characterization  of  SAR 
specifies  the  processes  underlying  the  reorienta¬ 
tion  stage. 

The  MFG  model  also  predicts  what  operators 
will  do  when  there  is  no  disruption  of  SA,  as 
when  there  is  no  interruption.  In  these  cases, 
attention  is  free  to  be  allocated  to  detecting 
changes  or  previously  unexplored  objects  in  the 
environment.  Therefore,  another  prediction  is 
that  attention  will  likely  be  directed  to  novel 
stimuli  when  SA  is  relatively  high  and  SAR  is 
not  necessary. 

Typically,  SA  is  measured  using  query  meth¬ 
ods.  The  situation  awareness  global  assessment 
technique  (SAGAT)  involves  freezing  the  com¬ 
puter  screen  and  probing  the  operator  with 
questions  related  to  the  operator’s  current  SA 
(Endsley,  2000).  Similarly,  the  situation  present 
assessment  method  (SPAM)  involves  probes  but 
administers  the  probe  online,  during  the  task, 
and  without  freezing  the  screen  (Durso  et  al., 
1995).  These  tools  can  measure  the  overall  state 
of  SA  or  the  level  of  SA  at  a  specific  moment  in 
time.  However,  these  tools  are  indirect  verbal 
measures  of  the  perceptual  processes  that  occur 
during  the  performance  of  a  dynamic  task. 

In  order  to  more  directly  measure  the  process 
of  SAR,  we  measured  operators’  eye  move¬ 
ments,  together  with  a  modified  SAGAT,  to  test 
our  predictions  regarding  visual  attention  and 
memory  during  SAR  versus  during  situations 


when  SA  is  relatively  high  and  SAR  is  less  nec¬ 
essary.  According  to  the  MFG  model,  an  inter¬ 
ruption  will  induce  behaviors  indicative  of  SAR, 
such  as  an  increased  resumption  lag  and  more 
fixations  on  objects  that  were  previously  looked 
at.  The  reason  for  this  result  is  that  the  interrup¬ 
tion  causes  the  episodic  codes  for  the  situation  to 
decay,  resulting  in  the  need  to  take  more  time  to 
examine  the  situation  and  a  greater  need  to  reac¬ 
tivate  the  goal  state  through  the  use  of  associa¬ 
tive  priming.  In  contrast,  when  SA  is  higher, 
such  as  during  continuous  task  performance, 
without  an  interruption,  it  is  hypothesized  that 
there  will  be  a  shorter  resumption  lag,  fewer 
refixations,  and  more  novel  object  fixation. 
There  will  be  more  novel  fixations  because 
attention  is  free  to  detect  new  events  in  the  envi¬ 
ronment  when  there  is  no  interruption.  This 
experiment  identifies  the  perceptual  and  cogni¬ 
tive  processes  that  characterize  SAR  by  induc¬ 
ing  SAR  through  immediate  interruptions  and 
negotiated  interruptions  and  compares  instances 
of  SAR  with  situations  in  which  SA  is  relatively 
high. 

EXPERIMENT  1 

To  investigate  SAR,  we  presented  partici¬ 
pants  with  short  videos  of  a  supervisory  control 
task  that  was  performed  by  another  operator.  We 
then  manipulated  whether  or  not  there  was  an 
immediate  interruption  during  the  playing  of  the 
video.  There  was  no  alert  before  the  interruption, 
so  preparation  was  presumably  minimal.  During 
the  interruption,  the  primary  task  was  hidden 
and  participants  performed  arithmetic  problems. 
Task  difficulty  was  also  manipulated  on  the 
primary  task  to  ensure  that  the  hypothesized 
behavior  is  found  in  a  wide  range  of  situations. 
After  each  trial,  participants  answered  questions 
that  were  adapted  from  the  SAGAT  and  designed 
to  assess  the  perception  and  comprehension 
components  of  SA.  Asking  questions  of  the  par¬ 
ticipant  after  each  trial  provided  a  measure  of  SA 
and  kept  the  participant  engaged. 

Method 

Participants.  Thirty  George  Mason  Univer¬ 
sity  undergraduate  students  participated  for  extra 
credit.  All  participation  was  voluntary.  One  par¬ 
ticipant  was  eliminated  due  to  instrument 
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Figure  1.  Research  Environment  for  Supervisory  Control  of  Heterogeneous  Unmanned  Vehicles  supervisory 
control  task. 


malfunction.  Two  participants  were  eliminated 
because  they  perfonned  at  or  below  chance  on 
the  SAGAT  questions.  In  total,  27  participants’ 
data  were  analyzed. 

Data  from  15  females  and  12  males  were  ana¬ 
lyzed.  The  average  age  of  participants  was  2 1 .7 
years  old  with  a  standard  deviation  of  3.6  years. 
Participants  were  asked  to  rate  how  often  they 
played  video  games  on  a  scale  of  1  (never),  2 
(sometimes),  or  3  (a  lot).  The  average  amount  of 
video  game  play  was  1 .9  with  a  standard  devia¬ 
tion  of  0.7.  All  participants  had  normal  or  cor- 
rected-to-normal  vision. 

Materials.  The  Research  Environment  for 
Supervisory  Control  of  Heterogeneous  Unmanned 
Vehicles  (RESCHU;  Boussemart  &  Cummings, 
2008)  was  the  supervisory  control  simulation 
used  in  this  study.  In  our  version  of  the  RESCHU 
task,  homogenous  unmanned  aerial  vehicles 


(UAVs)  moved  on  a  computer  screen  in  an  envi¬ 
ronment  that  was  dynamically  changing.  There 
were  three  main  sections  in  the  simulation:  the 
map  window  on  the  right,  the  payload  window 
on  the  upper  left,  and  a  status  window  on  the 
lower  left  (see  Figure  1).  The  map  area  displayed 
UAVs  (blue  half  ovals);  targets  (red  diamonds), 
toward  which  UAVs  were  directed  by  the  sys¬ 
tem;  and  hazards  (yellow  circles),  which  should 
be  avoided.  Vehicles  were  labeled  with  numbers 
and  targets  were  labeled  with  letters.  The  pay- 
load  window  (top  left)  displayed  a  visual  search 
photograph  image  in  which  the  participant  was 
directed  to  locate  a  target  object  based  on  writ¬ 
ten  instructions  as  part  of  a  payload  delivery 
operation  (described  later).  The  status  window 
(bottom  left)  depicted  a  timeline  of  each  UAV’s 
past  and  upcoming  milestones,  including  the 
waypoints  and  the  target  of  each  UAV. 
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The  simulation  included  five  UAVs  that 
moved  at  a  fixed  speed  throughout  the  duration 
of  the  task.  There  were  1 8  hazard  areas,  one  of 
which  changed  its  position  randomly  every  4  s, 
with  the  constraint  that  the  hazards  could  not 
appear  within  3°  of  visual  angle  (about  50  pix¬ 
els)  of  any  UAV.  If  the  UAV  passed  through  a 
hazard,  it  incurred  damage.  Damage  was  indi¬ 
cated  as  a  red  bar  in  the  status  window.  The  loca¬ 
tion  of  targets  and  hazards  on  the  simulation 
map  was  randomized  with  the  constraint  that 
targets  and  hazards  were  no  closer  than  3°  of 
visual  angle  from  each  other.  This  design 
ensured  that  targets  and  hazards  could  not  co¬ 
occur  in  the  same  position.  There  were  always 
seven  targets  present  on  the  map. 

The  participant  viewed  a  screen-capture 
video  of  a  session  of  an  operator’s  interaction 
with  the  RESCHU  simulation.  An  operator 
directed  UAVs  to  target  areas  while  avoiding 
hazard  areas.  To  avoid  a  hazard  area,  the  opera¬ 
tor  could  assign  the  UAV  to  a  different  target,  or 
the  operator  could  add  waypoints  to  the  UAV’s 
trajectory,  which  effectively  allowed  the  opera¬ 
tor  to  pilot  the  UAV  around  hazard  areas.  The 
operator  could  also  move  or  delete  waypoints. 
At  the  start  of  the  simulation,  the  UAVs  were 
randomly  assigned  to  targets  toward  which  they 
moved  along  automatically  generated  linear 
paths.  Once  the  UAV  reached  the  target  destina¬ 
tion,  the  target  flashed  red  until  it  was  engaged. 
A  target  was  engaged  when  the  operator  right- 
clicked  on  the  vehicle  and  selected  the  appropri¬ 
ate  pop-up  menu  item.  Engaging  the  vehicle 
triggered  the  payload  task,  whereby  the  operator 
performed  a  visual  search  task  to  identify  an 
object,  such  as  a  ship  or  a  car,  in  the  payload 
window. 

During  the  payload,  the  vehicles  in  the  map 
panel  continued  to  move  toward  their  respective 
targets,  and  operator  input  to  the  map  panel  was 
disallowed.  After  identifying  the  object  in  the 
payload  panel,  the  UAV’s  mission  was  com¬ 
pleted.  The  UAV  was  then  randomly  assigned  to 
a  new  target  that  did  not  already  have  a  UAV 
assigned  to  it. 

Each  simulation  session  had  unique  charac¬ 
teristics  with  randomly  generated  trajectories, 
locations,  and  objectives,  similar  to  a  real-world 
environment.  RESCHU  requires  the  operator  to 


manage  multiple  events  that  occur  in  parallel: 
More  than  one  UAV  could  be  waiting  at  a  target 
for  engagement,  multiple  UAVs  could  be  on  a 
path  to  a  hazardous  area,  and  it  was  left  to  the 
operator’s  discretion  to  act  on  any  one  of  the  five 
vehicles. 

Videos  of  RESCHU.  In  this  experiment,  the 
participant  was  not  the  operator  but  instead  the 
observer  of  a  recording  of  an  actual  human  oper¬ 
ator’s  interactive  session.  To  enable  a  high 
degree  of  experimental  control,  multiple  videos 
were  created  of  the  user  interface  while  the  oper¬ 
ator  performed  the  RESCHU  task.  Each  video 
clip  trial  was  8  s  long  and  did  not  include  a  pay- 
load.  There  were  two  versions  of  each  8-s  video 
clip.  One  version  included  the  full  8  s  without 
interruption,  whereas  in  the  other  version,  the 
8-s  video  was  cut  in  half  to  create  two  videos 
that  were  each  4  s  long  and  had  an  interruption 
between  them. 

The  video  clips  differed  based  on  the  number 
of  vehicles  that  needed  attention.  A  vehicle  needed 
attention  either  when  it  was  on  path  to  intersect  a 
hazard  or  when  the  vehicle  was  ready  to  be 
engaged.  There  were  10  unique  videos  in  the  prac¬ 
tice  task  and  72  unique  videos  in  the  experimental 
task.  Of  the  72  videos,  there  was  an  equal  number 
of  trials  in  which  one,  two,  or  three  vehicles 
needed  attention,  for  a  total  of  24  trials  for  each 
condition.  Half  of  the  video  clips  were  randomly 
assigned  to  the  interruption  condition,  wherein  an 
interruption  was  added  between  the  first  half  and 
the  second  half  of  the  video.  In  the  continuous- 
trials  condition,  the  8-s  video  clip  was  displayed 
without  interruption.  In  the  interruption  condition, 
the  participant  was  required  to  perform  math  prob¬ 
lems  during  the  break.  These  math  problems  con¬ 
sisted  of  three  simple  questions,  whereby  the  par¬ 
ticipant  was  required  to  add  three  single-digit 
numbers  together.  The  mean  duration  of  the  math 
problems  was  6,935  ms,  with  a  standard  deviation 
of  2,395  ms. 

The  video  clips  were  displayed  using  E-Prime 
experiment  software,  which  collected  user 
responses  to  questions  that  were  presented  after 
each  trial.  After  each  trial,  two  types  of  ques¬ 
tions  about  the  simulation  were  asked  of  the  par¬ 
ticipant.  The  questions  were  modeled  on  the 
SAGAT  (Endsley,  2000),  such  that  the  percep¬ 
tion  and  comprehension  components  of  SA 
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Perception 

1 .  Is  vehicle  X  idle? 

2.  Is  vehicle  X  on  path  to  hazard? 

Comprehension 

1 .  How  many  vehicles  are  on  path  to  hazard? 

2.  How  many  vehicles  are  idle? 

Figure  2.  Situation  awareness  global  assessment 
technique  questions  (Prefers  to  a  vehicle’s  numerical 
label). 

could  be  evaluated.  The  questions  also  kept  the 
participant  engaged  in  the  task.  After  each  trial, 
one  perception  question  and  one  comprehension 
question  were  randomly  presented  to  the  partici¬ 
pant.  The  perception  questions  involved  identi¬ 
fying  the  state  of  a  single  vehicle,  whereas  the 
comprehension  questions  required  the  partici¬ 
pant  to  integrate  information  about  all  the  vehi¬ 
cles  in  order  to  make  a  judgment  about  the  global 
situation  state  (see  Figure  2).  This  method  con¬ 
formed  to  Endsley’s  (1995b)  definition  of  the 
comprehension  stage  of  SA  as  the  “synthesis  of 
disjointed  elements.” 

Design  and  procedure.  The  experiment  had  a 
within-subjects  design  with  two  independent 
variables:  (a)  interruption  versus  no  interruption 
and  (b)  number  of  vehicles  needing  attention 
(one,  two,  three  vehicles),  resulting  in  six  differ¬ 
ent  types  of  video  clips,  which  were  presented  in 
random  order.  Each  participant  viewed  12  of 
each  of  the  six  types  of  video  clips. 

All  participants  began  the  experiment  by 
completing  an  interactive  tutorial  that  explained 
all  aspects  of  the  simulation.  Participants  learned 
about  the  objective  of  the  simulation:  to  prevent 
as  much  damage  as  possible  and  engage  as  many 
vehicles  as  possible.  Additionally,  participants 
learned  how  to  control  the  UAVs  (changing  tar¬ 
gets,  assigning/deleting/moving  waypoints)  and 
how  to  engage  a  target  (by  right-clicking  on  the 
target  and  selecting  the  engage  menu  item  in  the 
pop-up  menu).  Participants  were  also  warned  of 
the  dangers  of  hazards  and  were  instructed  on 
how  to  avoid  hazards.  The  tutorial  lasted  approx¬ 
imately  1 0  min. 

After  the  tutorial,  participants  were  instructed 
to  practice  interacting  with  the  RESCFIU  task 
until  they  understood  the  task  and  could  com¬ 
plete  each  subactivity.  Then  participants  were 


reminded  that  the  goal  of  the  task  was  to  prevent 
as  much  damage  as  possible  and  engage  as  many 
vehicles  as  possible.  Following  this  instruction, 
participants  were  exposed  to  5  min  practice  per¬ 
forming  the  RESCFIU  task.  Participants  were 
again  instructed  to  maximize  their  score  by 
engaging  as  many  targets  as  possible  and  pre¬ 
venting  as  much  damage  as  possible.  When  the 
simulation  ended,  participants  received  feed¬ 
back  on  how  many  vehicles  they  engaged  and 
the  total  amount  of  vehicle  damage. 

After  practicing  the  RESCHU  task,  partici¬ 
pants  were  asked  if  they  had  any  questions.  They 
were  then  given  instructions  on  the  videos  they 
were  about  to  view  of  the  RESCHU  task.  Par¬ 
ticipants  were  told  that  they  would  be  presented 
with  short  video  clips.  After  these  video  clips 
were  presented,  the  participant  was  asked  to 
answer  questions  about  the  clips.  As  is  conven¬ 
tional  with  the  SAGAT,  participants  were  given 
examples  of  all  of  the  questions  that  they  would 
be  expected  to  answer  about  the  video  clips.  Par¬ 
ticipants  were  also  told  that  some  of  the  videos 
would  be  interrupted  by  short  math  problems 
that  they  would  have  to  complete  as  quickly  and 
accurately  as  possible.  In  order  to  ensure  under¬ 
standing  of  the  task,  participants  were  given  10 
practice  trials.  After  completing  the  practice, 
participants  were  again  asked  if  they  had  any 
questions. 

Participants  were  then  seated  approximately 
66  cm  from  the  computer  monitor  and  were  cali¬ 
brated  on  an  eye  tracker.  Participants  then  began 
the  task.  The  72  videos  were  presented  in  ran¬ 
dom  order  to  participants.  The  order  of  each  type 
of  question  (i.e.,  perception  and  comprehension) 
was  also  randomized.  The  interruption  occurred 
randomly  on  half  of  the  trials.  After  the  task 
ended,  participants  were  debriefed. 

Measures.  Keystroke  data  were  collected  for 
each  participant  in  order  to  evaluate  their  responses 
to  the  SAGAT  questions.  Eye-tracking  data  were 
collected  using  an  SMI  eye  tracker  operating  at 
250  Hz.  A  fixation  was  defined  using  the  sac- 
cade -based  method,  whereby  fixations  occurred 
between  the  saccades.  A  saccade  was  defined  as 
an  eye  movement  velocity  faster  than  30°  per 
second. 

The  pattern  of  eye  movements  in  the  second 
half  of  the  video  clips  was  analyzed  in  order  to 
make  a  direct  comparison  between  instances  of 
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SAR  (i.e.,  when  there  was  an  interruption)  and 
instances  when  SA  was  likely  to  be  higher  (i.e., 
when  there  was  uninterrupted  task  performance). 
That  is,  the  interval  of  time  directly  after  the 
math  problems  was  compared  with  the  corre¬ 
sponding  interval  of  time  when  there  was  con¬ 
tinuous  presentation  of  the  video. 

Fixations  were  categorized  based  on  their 
object  of  focus.  There  were  a  total  of  five  UAVs 
on  the  screen,  each  having  a  different  target  and 
possibly  hazards  associated  with  it.  A  vehicle, 
the  vehicle’s  relevant  hazard(s),  and  the  vehi¬ 
cle’s  relevant  target  were  classified  as  a  “sepa¬ 
rate  vehicle  cluster.”  Since  there  were  a  total  of 
five  vehicles,  there  were  five  respective  separate 
vehicle  clusters.  A  fixation  on  an  object  was  cat¬ 
egorized  by  the  object’s  vehicle  cluster.  Fixa¬ 
tions  on  nonrelevant  hazards  and  targets  were 
removed  from  the  analysis. 

Separate  vehicle  cluster  fixations  were  fur¬ 
ther  characterized,  using  the  second  half  of  the 
video  clip,  as  either  refixations  or  novel  fixa¬ 
tions.  A  refixation  occurred  when  participants 
looked  at  the  same  vehicle  cluster  in  both  halves 
of  the  video  clip,  whereas  a  novel  fixation 
occurred  when  participants  looked  at  the  vehicle 
cluster  in  only  the  last  half  of  the  video  clip.  This 
categorization  was  then  used  as  an  independent 
variable  in  order  to  test  the  hypothesis  regarding 
the  types  of  fixations  that  occur  during  SAR.  For 
example,  during  the  first  half  of  the  video,  the 
participant  could  look  at  Vehicle  1,  Vehicle  2, 
and  Vehicle  3  (or  its  respective  target  or  hazard) 
and  then,  after  the  halfway  point,  fixate  again  on 
an  object  in  Vehicle  l’s  cluster.  That  fixation 
would  be  classified  as  a  refixation,  but  if  the  par¬ 
ticipant  then  fixated  on  Vehicle  5,  the  fixation 
would  be  classified  as  a  novel  fixation. 

Results 

Performance  on  the  modified  SAG  AT.  In  order 
to  determine  the  impact  of  interruptions  on  SA, 
accuracy  on  the  modified  SAGAT  was  compared 
in  the  interruption  and  no-interruption  conditions. 
In  order  to  determine  the  impact  of  the  number  of 
vehicles  that  needed  attention  on  SA,  accuracy  on 
the  modified  SAGAT  was  compared  when  one, 
two,  or  three  vehicles  needed  attention. 

Accuracy  was  determined  for  the  perception 
questions  and  the  comprehension  questions  by 
taking  the  average  percentage  correct  of  the 


two  questions  of  those  respective  types.  For  the 
perception  question,  there  was  no  significant 
difference  in  accuracy  between  the  interruption 
condition  (M  =  86.72%,  SD  =  8.58%)  and  con¬ 
tinuous  task  condition  (M  =  86.72%,  SD  = 
1 0.23%),  F(  1,26)  =  0.02, p  =  .89,  if  =  .00;  how¬ 
ever,  as  the  number  of  vehicles  that  needed 
attention  increased,  accuracy  declined,  F( 2,  52) 
=  10.57,  p  <  .05,  rf  =  .29.  Planned  contrasts 
showed  that  accuracy  on  the  perception  question 
was  worse  when  three  vehicles  needed  attention 
than  when  one  vehicle  needed  attention  ( p  <  .05) 
or  when  two  vehicles  needed  attention  ( p  <  .05). 
There  was  no  interaction  between  interruption 
and  number  of  vehicles  that  needed  attention, 
F(2,  52)  =  0.53,  p  =  .59,  r\  =  .02.  This  finding 
suggested  that  the  number  of  vehicles  that 
needed  attention  was  a  detenninant  of  the  diffi¬ 
culty  of  the  perception  stage  of  SA  but  that  inter¬ 
ruption  did  not  significantly  impact  the  percep¬ 
tion  stage  of  SA  (see  Figure  3). 

For  the  comprehension  questions,  there  was 
worse  accuracy  in  the  interruption  condition  (M  = 
74.79%,  SD  =  10.59%)  than  in  the  continuous  task 
condition  (M  =  84.05%,  SD  =  9.58%),  F(\,  26)  = 
32.41,/?  <  .05,  if  =  .55.  Additionally,  as  the  num¬ 
ber  of  vehicles  that  needed  attention  increased, 
accuracy  declined,  F(2,  52)  =  15.70,/?  <  .05,  r\2  = 
.38.  Planned  contrasts  showed  that  accuracy  on 
the  comprehension  question  was  better  when  one 
vehicle  needed  attention  than  when  three  vehicles 
needed  attention  (p  <  .05),  and  accuracy  on  the 
comprehension  question  was  better  when  two 
vehicles  needed  attention  than  when  three  vehi¬ 
cles  needed  attention  (p  <  .05).  There  was  no 
interaction  between  interruption  and  number  of 
vehicles  that  needed  attention,  F( 2,  52)  =  1.01, 
p  =  .37,  rf  =  .04  (see  Figure  3). 

Thus,  interruptions  did  not  seem  to  signifi¬ 
cantly  impact  the  perceptual  stage  of  SA  but  had 
a  detrimental  impact  on  the  comprehension 
stage  of  SA.  However,  the  number  of  vehicles 
needing  attention  had  a  detrimental  impact  on 
both  stages  of  SA. 

Number  of  fixations.  Whereas  the  question¬ 
naire  assessed  SA,  fixations  provided  insight 
into  participants’  strategies  for  compensating  for 
reduced  SA.  One  manner  to  compensate  for 
reduced  SA  is  to  increase  scanning  in  an  attempt 
to  restore  SA.  In  order  to  determine  if  there  were 
differences  in  the  number  of  fixations  between 
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Perception  Accuracy 


Comprehension  Accuracy 


Figure  3.  Accuracy  on  situation  awareness  global  assessment  technique 
questions  as  a  function  of  the  number  of  vehicles  needing  attention  and 
of  interruption  versus  no-interruption  condition  (i.e.,  the  continuous  task 
condition).  Error  bars  are  95%  confidence  intervals. 


the  interruption  condition  and  the  continuous  separate  vehicle  cluster  fixations  that  occurred 
task  condition,  we  examined  the  number  of  in  the  second  4  s  of  each  trial.  The  dependent 
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variable  was  the  number  of  separate  vehicle 
cluster  fixations;  thus  the  number  of  separate 
vehicle  cluster  fixations  was  between  0  and  5, 
since  there  were  five  vehicles  in  the  simulation. 
There  was  a  marginal  effect  of  interruption  on 
the  number  of  separate  vehicle  cluster  fixations 
in  the  second  half  of  the  videos,  with  more  vehi¬ 
cle  cluster  fixations  in  the  interruption  condition 
(M  =  3.15,  SD  =  0.71)  than  in  the  continuous 
task  condition  (M  =  3.01,  SD  =  0.71),  t( 26)  = 
3.68, p  =  .07,  d  =  .20. 

Fixation  duration.  One  unexpected  differ¬ 
ence  between  the  interruption  condition  and  the 
continuous  task  condition  was  that  fixation 
duration  for  all  relevant  objects  was  different 
between  conditions  in  the  second  half  of  the 
video  trial.  Fixation  duration  in  the  second  half 
of  each  trial  was  shorter  in  the  interruption  con¬ 
dition  ( M  =  196  ms,  SD  =  43  ms)  than  in  the 
continuous  task  condition  (M  =216  ms,  SD  = 
54  ms),  t( 26)  =  3.72, p  <  .05,  d=  .41.  This  find¬ 
ing  may  explain  why  there  was  a  marginal  effect 
of  number  of  fixations,  with  more  fixations  in 
the  interruption  condition.  During  SAR,  partici¬ 
pants  may  have  quickly  scanned  the  screen  in 
order  to  recover  awareness  of  the  key  elements 
involved  in  the  situation.  Perhaps  participants 
made  shorter  fixations  in  the  interruption  condi¬ 
tion  in  an  effort  to  examine  more  objects  (i.e., 
they  made  more  fixations). 

Novel  fixations  versus  refixations.  Recall 
that  MFG  predicts  that  there  will  be  more  refix¬ 
ations  after  an  interruption  for  the  purpose  of 
reactivating  decayed  memory  codes  through 
associative  priming.  Additionally,  it  was  pre¬ 
dicted  that  there  would  be  more  refixations  in 
the  interruption  condition  because  refixations 
can  prime  the  activation  of  decaying  memory 
traces.  We  used  a  within-groups  ANCOVA  to 
examine  the  relationship  between  fixation  type 
(novel,  refixations)  and  interruption  (interrup¬ 
tion,  continuous  task)  on  the  dependent  variable 
of  number  of  separate  vehicle  cluster  fixations  in 
the  second  half  of  the  8-s  trials.  Furthermore,  we 
included  the  number  of  vehicles  that  needed 
attention  as  a  covariate.  After  controlling  for  the 
number  of  vehicles  that  needed  attention,  there 
was  no  difference  in  the  number  of  separate 
vehicle  cluster  fixations  in  the  interruption  con¬ 
dition  (M  =  1.56,  SD  =  0.98)  compared  to  the 


continuous  task  condition  (M=  1 .50,  SD  =  0.83), 
F(l,26)  =  3.59,/?  =  .12,  rf  =  .29.  However,  there 
was  a  main  effect  of  fixation  type  after  control¬ 
ling  for  the  number  of  vehicles  that  needed 
attention,  F(  1,  26)  =  77.22,  p  <  .05,  ip  =  .75, 
with  more  separate  vehicle  cluster  fixations  on 
vehicles  that  were  previously  looked  at,  that  is, 
refixations  (M=  2.21,  SD  =  0.78),  than  on  vehi¬ 
cles  that  had  not  previously  been  looked  at,  that 
is,  novel  fixations  (M=  0.85,  SD  =  0.29). 

Consistent  with  MFG,  there  was  a  two-way 
interaction  between  fixation  type  and  interrup¬ 
tion  after  controlling  for  the  number  of  vehicles 
that  needed  attention,  F(  1,  26)  =  18.43,/?  <  .05, 
rf  =  .42.  In  planned  contrasts,  all  groups  were 
significantly  different  from  all  other  groups 
(p  <  .05),  with  the  exception  of  a  marginal  effect 
for  the  comparison  between  novel  fixations  in 
the  interruption  condition  versus  the  continuous 
task  condition  (/?  =  .07).  As  Figure  4  suggests, 
SAR  was  distinguished  by  more  refixation  in  the 
interruption  condition  than  in  the  continuous 
task  condition  and  marginally  fewer  novel  fixa¬ 
tions  in  the  interruption  condition  than  in  the 
continuous  task  condition. 

Discussion 

We  sought  to  decrease  SA  in  order  to  under¬ 
stand  SAR  by  manipulating  whether  or  not 
participants  were  exposed  to  immediate  inter¬ 
ruptions.  We  predicted  that  interruptions  would 
undermine  the  perception  and  comprehension 
components  of  SA.  A  decreased  level  of  SA 
requires  participants  to  engage  in  SAR  in  order 
to  refresh  their  memory.  The  MFG  model  pre¬ 
dicts  that  during  SAR,  participants  will  refixate 
on  objects  that  were  previously  looked  at  in 
order  to  prime  their  memory  of  the  environ¬ 
ment.  However,  when  there  is  no  interruption, 
SA  is  relatively  higher,  enabling  the  participant 
to  allocate  attention  to  detect  novel  elements  in 
the  environment. 

Since  there  were  separate  measures  of  SA  and 
SAR,  reduced  SA  could  be  related  to  specific  SAR 
behaviors.  Interestingly,  perception  SAdid  not  dif¬ 
fer  between  the  interruption-versus-continuous 
conditions,  but  comprehension  SA  was  worse  for 
the  interruption  condition  than  for  the  continuous 
condition.  Comprehension  is  a  later  stage  of  SA 
than  perception  because  it  is  necessary  to  perceive 
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Re-fixation  vs  Novel  Fixation  Based  on  Task  Condition 


Task  Break  Continuous  Task 


Figure  4.  Type  of  fixation  based  on  task  condition.  Error  bars  are  95% 
confidence  intervals. 


elements  in  an  environment  before  their  meaning 
can  be  comprehended.  For  example,  it  is  impossi¬ 
ble  to  determine  that  three  vehicles  are  on  path  to 
intersect  with  a  hazard  before  first  looking  at  each 
vehicle,  analyzing  its  trajectory,  and  identifying 
that  they  will  intersect  with  a  hazardous  area. 
Since  comprehension  SA  depends  on  first  acquir¬ 
ing  perception  SA,  comprehension  SA  takes  lon¬ 
ger  to  be  reacquired,  and  therefore  it  might  be 
more  vulnerable  to  disruption  than  perception  SA. 
The  finding  that  participants  perfonned  worse  on 
SA  comprehension  after  an  interruption  explains 
why  they  would  need  to  do  more  SAR  when  there 
was  an  interruption  than  when  there  was  no  inter¬ 
ruption. 

The  eye-tracking  data  supported  our  MFG- 
based  hypothesis  regarding  where  attention  is 
allocated  during  SAR.  After  an  interruption, 
during  SAR,  participants  fixated  more  on  objects 
that  were  previously  looked  at  and  fixated  on 
relevant  novel  objects  marginally  less  than  dur¬ 
ing  continuous  task  performance.  MFG  supports 


this  nuanced  finding  because  MFG  predicts  that 
after  a  break,  the  goal  representation  needs  to  be 
reinstated  through  either  internal  (memory/ 
imagination)  or  external  (fixation)  cues.  In  sup¬ 
port  of  this,  participants  fixated  on  objects  that 
were  previously  gazed  on  after  an  interruption  in 
order  to  reinstate  their  previous  memory  repre¬ 
sentation.  We  interpret  this  finding  as  indicating 
that  participants  use  contextual  cues  to  increase 
activation  of  the  memory  state  during  SA  recov¬ 
ery.  In  contrast,  during  continuous  task  execu¬ 
tion,  when  there  is  no  interruption,  participants 
had  less  need  for  SAR  because  they  preserved 
their  SA.  Thus,  when  there  was  no  interruption, 
participants  spent  marginally  more  time  seeking 
out  novel  aspects  of  the  environment. 

In  addition  to  characterizing  the  type  of  fixa¬ 
tions  involved  in  SAR,  our  data  revealed,  sur¬ 
prisingly,  that  fixations  were  of  shorter  duration 
when  SA  was  disrupted  by  an  interruption  than 
when  SA  was  not  disrupted.  Faster  fixation 
durations  and  a  marginal  increase  in  the  number 
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of  fixations  in  the  second  half  of  the  interruption 
condition  suggest  that  a  characteristic  of  SAR  is 
to  cause  someone  to  scan  the  environment  more 
thoroughly  by  looking  at  objects  for  a  shorter 
duration  of  time. 

In  sum,  we  identified  a  new  concept,  SAR, 
and  found  perceptual  and  cognitive  processes 
that  characterize  SAR,  including  more  refix¬ 
ations  and  shorter  fixation  durations.  However, 
in  Experiment  1,  we  could  not  test  predictions 
about  the  hypothesized  resumption  lag  after  an 
interruption  because  trial  duration  was  fixed  and 
there  was  no  timed  response.  Additionally,  there 
were  task  constraints  in  Experiment  1  that  may 
explain  the  findings,  such  as  the  fact  that  partici¬ 
pants  passively  watched  videos  and  answered 
questions  related  to  those  videos,  instead  of 
actively  engaging  in  the  RESCHU  task.  SA  may 
be  lower  when  passively  viewing  a  task,  and 
passive  viewing  may  impact  behavior  (Endsley 
&  Rodgers,  1998).  Additionally,  SAR  may  differ 
under  less  constrained  scenarios;  for  instance, 
flexible  trial  durations  permit  the  participant  to 
more  fully  scan  the  environment;  and  enabling 
the  participant  to  choose  when  he  or  she  is  inter¬ 
rupted,  by  administering  a  negotiated  interrup¬ 
tion  instead  of  an  immediate  interruption,  may 
mitigate  the  need  for  SAR.  The  task  constraints 
and  the  environment  can  be  driving  forces  that 
affect  operator  SA  (Kirlik  &  Strauss,  2006; 
Smith  &  Hancock,  1995),  and  these  constraints 
may  explain  the  findings  regarding  SAR.  To 
address  these  issues,  and  to  determine  if  our 
findings  generalize  to  a  more  dynamic  and 
actively  engaged  task,  we  conducted  another 
experiment  to  determine  if  the  perceptual  and 
cognitive  processes  that  characterized  SAR  in 
this  experiment  are  replicated  when  participants 
perform  the  RESCHU  simulation. 

EXPERIMENT  2 

Experiment  1  offered  support  for  the  hypoth¬ 
esized  cognitive  and  perceptual  processes  that 
underlie  SAR.  To  determine  whether  the  find¬ 
ings  of  Experiment  1  occur  when  participants  are 
actively  engaged  in  a  dynamic  task,  and  when 
the  participant  can  control  the  timing  of  an  inter¬ 
ruption,  participants  performed  the  RESCHU 
task  under  either  a  negotiated  interruption  or 


continuous  task  condition.  Participants  had  some 
control  over  exactly  when  their  secondary  task 
occurred  because  engaging  the  vehicle  caused 
the  interruption  to  begin.  The  interruption  was 
a  visual  search  task  that  involved  delivering  a 
payload  (upper  left  corner  of  Figure  1).  This 
payload  delivery  task  was  the  negotiated  inter¬ 
ruption  to  the  main  task  in  the  map  panel.  In  the 
continuous  task  condition,  no  interrupting  pay- 
load  delivery  task  was  required  after  engaging  a 
vehicle.  Another  difference  between  Experiment 
2  and  Experiment  1  was  that  during  the  payload, 
the  main  task  was  not  suspended.  We  hypoth¬ 
esized  that  an  interruption  will  impair  SA  and,  as 
a  result,  induce  the  process  of  SAR.  Also,  since 
trial  durations  were  not  fixed  in  Experiment  2,  it 
was  possible  to  assess  task  resumption  time.  We 
predicted  that  a  characteristic  of  SAR  would  be 
increased  task  resumption  time. 

Method 

Participants.  Eighty-one  George  Mason  Uni¬ 
versity  undergraduate  students  participated  for 
extra  credit.  All  participation  was  voluntary  and 
participants  had  no  prior  experience  with  the 
task.  Eye  data  for  3  participants  were  eliminated 
because  it  was  not  accurately  captured.  In  total, 
78  participants  were  analyzed:  38  in  the  inter¬ 
ruption  condition  and  40  in  the  continuous  task 
condition. 

Data  from  58  females  and  20  males  were  ana¬ 
lyzed.  The  average  age  of  participants  was  20.4 
years  old  with  a  standard  deviation  of  3.0  years. 
Participants  were  asked  to  rate  how  often  they 
played  video  games  on  a  scale  of  1  (never),  2 
(sometimes),  or  3  (a  lot).  The  average  amount  of 
video  game  play  was  1.8  with  a  standard  devia¬ 
tion  of  0.7.  All  participants  had  normal  or  cor- 
rected-to-normal  vision. 

Materials.  Materials  were  identical  to  the 
RESCHU  task  described  in  Experiment  1,  with 
the  exception  that  the  premade  videos  of  the 
RESC1TU  task  were  not  used.  Instead,  partici¬ 
pants  interacted  with  the  simulation. 

Design  and  procedure.  The  experiment  had  a 
between-groups  design.  One  group  was  assigned 
to  the  continuous  task  condition,  and  the  other 
group  was  assigned  to  the  interruption  condition. 
The  interruption  consisted  of  a  visual  search  pay- 
load  task  following  vehicle  engagement.  The 
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dependent  variables  were  patterns  of  eye  fixa¬ 
tions  directly  after  the  participant  completed  a 
vehicle  engagement  (continuous  task  condition) 
or  else  after  they  completed  a  vehicle  engage¬ 
ment  followed  by  a  visual  search  payload  task 
(interruption  condition),  which  constituted  an 
interruption  from  the  primary  task.  The  average 
duration  of  the  visual  search  payload  task  inter¬ 
ruption  was  6,307  ms,  with  a  standard  deviation 
of  1,504  ms.  By  converting  the  time  of  the  inter¬ 
ruption  to  distance  traveled,  we  found  that  on 
average,  the  vehicles  moved  33  pixels  during  the 
visual  search  payload  task,  with  a  standard  devi¬ 
ation  of  8  pixels.  Once  engaged  in  the  visual 
search  payload  task,  the  participant  was  unable 
to  complete  any  other  actions  in  the  simulation 
until  the  visual  search  task  was  completed. 

The  procedure  was  identical  to  Experiment  1 
except  that  after  the  practice,  participants  used 
the  RESCHU  simulation.  Participants  were  cali¬ 
brated  on  the  eye  tracker,  were  seated  approxi¬ 
mately  66  cm  from  the  screen,  were  told  to  try  to 
avoid  damage  as  much  as  possible  and  to  engage 
as  many  vehicles  as  possible,  and  then  were 
administered  a  10-min  session  on  RESCE1U. 
This  session  was  followed  by  a  brief  break,  after 
which  a  second  10-min  RESCHU  session  was 
administered  in  the  same  manner  as  the  first. 
Participants  were  run  in  the  same  condition  for 
both  10-min  sessions,  and  both  sessions  were 
combined  in  the  analysis. 

Measures.  The  eye  movement  measures  were 
identical  to  Experiment  1,  except  that  the  interval 
of  time  that  was  analyzed  was  different.  We 
parsed  the  session  into  discrete  events  in  a  simi¬ 
lar  manner  as  Crandall,  Goodrich,  Olsen,  and 
Nelson  (2005)  and  Altmann  and  Trafton  (2004), 
by  dividing  the  task  into  intervals  in  which  either 
(a)  participants  monitor  the  screen  and  decide 
what  to  do  next  or  (b)  participants  perform 
actions.  The  monitoring  interval  of  particular 
interest  was  the  interval  that  began  after  an 
engagement  completion.  This  interval  immedi¬ 
ately  followed  the  interruption,  consisting  of  a 
visual  search  task  in  the  task-break  condition  or 
else  immediately  followed  target  engagement  in 
the  continuous  condition.  Examining  the  moni¬ 
toring  interval  after  the  mission  completion  and 
before  the  beginning  of  the  next  action,  whether 
an  engagement  or  a  hazard  evasion  action, 
enabled  us  to  compare  functionally  equivalent 


intervals  in  the  two  conditions.  The  duration  of 
this  monitoring  interval  differed  depending  on 
how  long  it  took  the  participant  to  initiate  the 
next  action. 

Results 

All  analyses  were  conducted  on  the  monitor¬ 
ing  interval  of  time  after  the  mission  comple¬ 
tion  and  before  the  next  action.  On  average, 
participants  completed  54.3  missions  across  the 
two  1 0-min  sessions,  resulting  in  an  average  of 
54.3  monitoring  intervals  that  were  analyzed  per 
participant. 

Resumption  lag.  Recall  that  MFG  predicts 
that  after  an  interruption,  participants  will  take 
longer  to  initiate  their  next  action,  which  is 
known  as  the  resumption  lag.  According  to 
MFG,  participants  take  longer  to  respond  after 
an  interruption  because  it  causes  episodic  codes 
for  the  situation  to  decay.  In  response  to  this 
decay,  the  participant  must  refixate  on  objects  in 
order  to  reactivate  the  goal  state  through  asso¬ 
ciative  priming.  Consistent  with  MFG,  partici¬ 
pants  in  the  interruption  condition  experienced  a 
longer  resumption  lag;  they  took  longer  to 
resume  (M=  5,033  ms,  SD  =  1,145  ms)  than  did 
participants  in  the  continuous  task  condition 
(M  =  4,124  ms,  SD  =  1,122  ms),  f( 76)  =  3.54, 
p  <  .05,  d  =  .80.  This  finding  is  consistent  with 
other  research  that  has  shown  increased  resump¬ 
tion  lag  after  an  interruption  (Hodgetts  &  Jones, 
2006;  Monk  et  al.,  2004;  Trafton  et  al.,  2003). 

Fixation  duration.  Consistent  with  the  find¬ 
ings  from  Experiment  1,  fixation  durations  in 
the  interruption  condition  were  shorter  (M=  314 
ms,  SD  =  69  ms)  than  in  the  continuous  task  con¬ 
dition  ( M  =  404  ms,  SD  =  82  ms),  t{ 76)  =  5.24, 
p  <  .05,  d  =  1.19.  This  finding  suggested  again 
that  during  SAR,  participants  quickly  scan 
objects  in  the  environment  in  order  to  sample 
objects  in  the  enviromnent  and  reactivate  their 
awareness  for  those  objects. 

Novel  fixations  versus  refixations .  In  Experi¬ 
ment  1,  there  was  an  interaction  between  fixa¬ 
tion  type  and  task  condition,  with  more 
refixations  in  the  interruption  condition  and 
marginally  more  novel  fixations  in  the  continu¬ 
ous  task  condition.  To  explore  whether  this 
effect  occurs  when  actively  engaged  in  a  dynamic 
task,  we  ran  a  mixed  ANOVA  to  examine  the 
relationship  between  novel  fixation  or  refixation 
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Re-fixation  vs  Novel  Fixation  Based  on  Task  Condition 


Task  Break  Continuous  Task 


Figure  5.  Type  of  fixation  based  on  task  condition.  Error  bars  are  95% 
confidence  intervals. 


and  continuous  task  or  interruption.  Task  condi¬ 
tion  (interruption  or  continuous  task)  was  the 
between-groups  factor  and  fixation  type  (refix- 
ation  or  novel  fixation)  was  the  within-groups 
factor.  The  dependent  measure  was  the  number 
of  separate  fixation  clusters  that  occurred  in  the 
monitoring  interval  after  the  mission  completion 
and  before  the  next  action. 

Consistent  with  MFG,  there  was  a  main  effect 
of  task  condition,  with  more  separate  vehicle 
cluster  fixations  in  the  interruption  condition 
(M=  1.29,  SD  =  0.36)  than  in  the  continuous  task 
condition  (M=  1 .02,  SD *  0.2 1 ),  F(1 , 76)  =  40.55, 
p  <  .05,  q2  =  .40.  This  finding,  together  with  the 
finding  of  shorter  fixation  duration,  suggests  that 
the  interruption  resulted  in  increased  scanning 
due  to  quicker  fixations  and  more  fixations,  pre¬ 
sumably  in  the  service  of  SAR.  There  was  a  mar¬ 
ginal  main  effect  of  fixation  type  with  more  refix- 
ations  (M=  1.19,  SD  =  0.49)  than  novel  fixations 
(M=  \  .\2,SD  =  0.09),  F{ 1, 76)  =  3.27,  p  =  .07,  q2 
=  .04.  Also  consistent  with  the  MFG  model  of 


SAR,  there  was  an  interaction  between  task  con¬ 
dition  and  fixation  type,  F(  1 , 76)  =  49.72,/?  <  .05, 
q2  =  .40.  All  groups  were  significantly  different 
from  all  other  groups  (p  <  .05  using  planned  con¬ 
trasts),  with  the  exception  of  the  difference 
between  the  number  of  novel  separate  fixations  in 
the  continuous  task  condition  versus  the  interrup¬ 
tion  condition  (p  =  .  1 7).  As  Figure  5  suggests,  in 
the  interruption  condition,  there  were  more  refix- 
ations  than  novel  fixations,  but  in  the  continuous 
task  condition,  there  were  marginally  more  novel 
fixations  than  refixations. 

Discussion 

In  this  experiment,  SAR  was  identified  by 
comparing  the  cognitive  and  perceptual  pro¬ 
cesses  that  occurred  in  an  interruption  condition 
with  those  that  occurred  in  a  continuous  task 
condition,  without  an  interruption.  After  the 
interruption,  participants  took  longer  to  initiate 
the  next  action,  were  more  likely  to  refixate  on 
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TABLE  1 :  A  Summary  of  the  Characteristics  of  Situation  Awareness  Recovery  (SAR)  and  the  Memory- 
for-Goals  (MFG)  Explanation 

Characteristics  of  SAR 

MFG  Explanation 

More  refixations  and  fewer  novel  fixations 

Longer  resumption  lag 

Shorter  fixations  and  more  fixations 

Refixations  act  as  contextual  cues  that  prime  memory  for 
the  situation  and  thereby  boost  situation  awareness  (SA). 

Participants  must  spend  time  recovering  SA  before  they 
can  decide  what  action  to  take  next. 

In  an  effort  to  reactivate  degraded  memory  traces, 
participants  sample  more  of  the  environment. 

previously  viewed  objects,  made  shorter  fixa¬ 
tions,  and  made  more  fixations. 

MFG  was  supported  by  the  pattern  of  fixa¬ 
tions.  In  the  interruption  condition,  SAR  activi¬ 
ties  included  fixating  on  previously  viewed 
environmental  cues,  presumably  in  the  service 
of  reactivating  memory  traces,  more  than  on 
novel  objects.  Additionally,  in  the  continuous 
task  condition,  which  we  assume  required  less 
SAR,  participants  had  more  fixations  on  novel 
objects  than  on  previously  viewed  objects. 
Experiment  2  also  replicated  the  finding  in 
Experiment  1  that  participants  made  shorter- 
duration  fixations  when  performing  SAR  and,  in 
contrast  to  Experiment  1,  made  significantly, 
rather  than  marginally,  more  fixations  during 
SAR,  suggesting  an  effort  to  recover  SA  by 
more  thoroughly  scanning  the  environment. 

A  prediction  of  MFG  is  that  participants  take 
longer  to  respond  after  an  interruption  due  to  the 
need  to  reactivate  decayed  episodic  memory 
chunks.  In  support  of  this  hypothesis,  partici¬ 
pants  took  longer  to  take  their  next  action  after 
an  interruption  than  under  continuous  task  per¬ 
formance.  This  increased  resumption  lag  may 
also  explain  why  there  were  more  fixations  after 
an  interruption  than  after  continuous  task  perfor¬ 
mance.  Therefore,  in  Experiment  2,  we  replicated 
our  findings  from  Experiment  1  and  identified 
additional  perceptual  processes  that  characterize 
SAR  in  a  more  realistic  task,  which  included 
increased  resumption  lag  and  more  fixations 
(see  Table  1). 

GENERAL  DISCUSSION 

Two  methodologies  were  used  to  identify 
the  perceptual  and  cognitive  processes  that 


characterize  SAR.  In  Experiment  1 ,  participants 
viewed  short  video  clips  of  the  RESCHU  super¬ 
visory  control  simulation  and  either  experienced 
an  interruption  halfway  through  the  video  or 
experienced  a  continuous  clip  without  an  inter¬ 
ruption.  In  Experiment  2,  participants  actively 
engaged  in  the  RESCHU  simulation  and  were 
assigned  to  either  an  interruption  scenario  in 
which  engaging  the  vehicle  was  followed  by  a 
visual  search  interruption  or  a  scenario  in  which 
no  interruption  occurred.  In  both  experiments, 
the  time  interval  after  the  interruption  was  the 
focus  of  the  analysis. 

In  Experiment  1,  perceptual  and  comprehen¬ 
sion  SA  were  assessed  by  a  SAGAT-based  ques¬ 
tionnaire.  Although  there  was  no  difference  in 
perception  SA  between  the  interruption  and  con¬ 
tinuous  conditions,  comprehension  SA  was 
significantly  impaired  by  the  interruption.  This 
finding  suggested  that  later  stages  of  SA  are 
more  susceptible  to  disruption  and  that  interrup¬ 
tions  induce  the  need  for  SA  to  be  recovered. 
This  result  meant  that  we  could  test  our  predic¬ 
tions  regarding  SAR  by  comparing  the  behav¬ 
iors  in  the  second  halves  of  the  video  clips  for 
the  two  conditions. 

Our  predictions  concerning  SAR  were  based 
on  the  MFG  theory.  Although  Endsley’s  (1995b) 
model  describes  at  a  high  level  the  role  of  goal- 
directed  processing  in  guiding  attention,  it  does 
not  make  predictions  about  how  participants 
activate  goals  in  specific  situations.  Using  MFG 
allowed  additional,  fine-grained  process  predic¬ 
tions  to  be  made  in  a  dynamic  task.  In  the  MFG 
model,  the  loss  of  SA  is  due  to  decayed  memory 
traces  for  previous  goals,  which  can  occur  even 
after  very  short  interruptions  (Altmann,  Trafton, 
&  Hambrick,  in  press),  and  thus  SA  recovery 
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following  even  these  brief  interruptions  involves 
the  reactivation  of  the  memory  traces.  Memory 
traces  are  reactivated  through  the  search  for 
memory  cues,  for  example,  objects  that  are 
related  to  decayed  goals.  Eye  movements  pro¬ 
vided  evidence  of  this  sort  of  SAR  search, 
including  short-duration  fixations  and  refix¬ 
ations  on  objects  that  had  previously  been  exam¬ 
ined. 

Experiment  2  replicated  the  findings  from 
Experiment  1  in  the  context  of  active  user  inter¬ 
action  with  the  simulation  environment  and 
revealed  additional  behaviors  that  characterize 
SAR.  These  findings  included  an  increased 
number  of  fixations,  a  longer  resumption  lag, 
and  fewer  fixations  on  novel  objects.  These  find¬ 
ings  suggested  that  participants  sample  the  envi¬ 
ronment  more  during  SAR  in  an  effort  to 
increase  activation  for  decayed  memories  and 
goals. 

Taken  together,  these  experiments  identified  a 
new  concept:  SAR.  SAR  is  a  process  whereby  SA 
is  improved  after  it  has  been  degraded.  SAR  was 
characterized  by  more  fixations,  a  longer  resump¬ 
tion  lag,  fixations  that  were  shorter  in  duration, 
more  refixations,  and  fewer  novel  fixations. 

These  findings  are  consistent  with  the  MFG 
model.  The  MFG  model  predicts  that  there  will 
be  increased  resumption  lag  after  an  interruption 
due  to  the  reactivation  of  decayed  memory 
elements  (Trafton  et  al.,  2003).  Although  this 
prediction  could  not  be  explicitly  tested  in 
Experiment  1 ,  in  Experiment  2,  resumption  lag 
increased  after  an  interruption.  Additionally,  the 
priming  constraint  of  the  MFG  model  states  that 
an  episodic  code  can  be  primed  by  contextual 
retrieval  cues,  in  the  present  context,  provided 
by  the  relevant  stimulus  objects  (vehicles,  haz¬ 
ards,  targets;  Altmann  &  Trafton,  2002;  Ratwani, 
Andrews,  Sousk,  &  Trafton,  2008).  Consistent 
with  this  hypothesis,  we  found  in  both  Experi¬ 
ment  1  and  Experiment  2  increased  scanning  of 
previously  viewed  objects  after  an  interruption, 
suggesting  the  use  of  priming  as  a  strategy  to 
regain  SA.  Such  activity  appears  to  represent  a 
method  to  recover  SA  after  the  break. 

How  SA  is  recovered  in  a  given  context  is 
likely  related  to  the  degree  and  nature  of  impair¬ 
ment  to  SA.  We  observed  preferential  looking  at 
previously  viewed  objects  during  SAR,  imply¬ 
ing  that  participants  retained  some  memory  for 


those  objects  after  the  break  but  needed  to  reac¬ 
tivate  their  encoding  of  those  objects.  Consistent 
with  this  interpretation,  the  interruption  did  not 
interfere  with  lower-level  perceptual  SA,  as 
assessed  by  the  SAGAT  questionnaire  in  Experi¬ 
ment  1 ,  but  only  interfered  with  the  comprehen¬ 
sion  stage  of  SA.  Therefore,  we  examined  SAR 
whereby  participants  retained  some  memory  of 
the  situation,  so  our  findings  may  not  generalize 
to  situations  in  which  SA  is  completely  lost. 

Another  limitation  of  these  experiments  was 
that  SA  was  evaluated  only  with  the  SAGAT  in 
Experiment  1  and  was  evaluated  only  once  at  the 
end  of  each  trial.  However,  the  convention  is  to 
administer  the  SAGAT  at  random  times  during  a 
trial.  The  reason  that  we  administered  the 
SAGAT  at  the  end  of  each  trial  was  to  ensure 
that  the  SAGAT  did  not  interfere  with  process¬ 
ing  during  the  trial.  However,  we  could  infer  that 
SA  was  reduced  in  Experiment  2  based  on  our 
findings  from  Experiment  1  and  prior  research 
regarding  the  disruptive  effects  of  interruptions 
(Altmann  &  Gray,  2008;  Monk  et  al.,  2004;  Rat¬ 
wani  et  al.,  2008;  Trafton  et  al.,  2003).  Nonethe¬ 
less,  future  research  can  include  direct  measures 
of  SA  throughout  the  task  in  order  to  directly 
identify  how  the  amount  of  SA  reduction  affects 
the  process  of  SAR. 

An  understanding  of  SAR  can  be  used  to  mit¬ 
igate  operator  errors,  since  errors  are  highly 
associated  with  reduced  SA  (Hartel  et  al.  1991). 
If  situations  in  which  the  operator  has  low  SA 
can  be  distinguished  and  the  operator  does  not 
engage  in  SAR,  these  are  instances  in  which  the 
operator  is  more  likely  to  make  an  error.  Since 
not  engaging  in  SAR  when  it  is  needed  can 
result  in  increased  errors,  it  may  be  useful  to 
invoke  adaptive  automation  in  these  instances. 
For  example,  participants  can  be  cued  to  look  at 
previously  looked  at  objects  in  order  to  improve 
their  SA  through  associative  priming. 
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KEY  POINTS 

•  Situation  awareness  recovery  (SAR)  is  the  pro¬ 
cess  of  regaining  situation  awareness  (SA)  after 
the  operator  becomes  distracted  or  following  an 
interruption. 

•  The  memory-for-goals  model  (Altmann  &  Traf- 
ton,  2002)  makes  specific  predictions  about  how 
operators  will  engage  in  SAR:  The  priming  con¬ 
straint  posits  that  cues  in  the  environment  will 
increase  activation  for  degraded  memory  elements 
through  spreading  activation. 

•  The  perceptual  and  cognitive  processes  involved  in 
SAR  include  refixating  on  cues  in  the  environment, 
increased  resumption  lag,  increased  scanning,  and 
shorter-fixation  durations.  These  results  are  highly 
consistent  with  the  memory-for-goals  theory. 
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